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ABSTRACT 

A survey of line pilots' attitudes about flight deck automation was conducted by the Royal Air Force Institute of 
Aviation Medicine (RAF IAM, Farnborough, UK) under the sponsorship of the United Kingdom’s Civil Aviation 
Authority and in cooperation with IATA (the International Air Transport Association). Survey freehand comments 
given by pilots operating 13 types of commercial transports across five manufacturers (Airbus, Boeing, British 
Aerospace, Lockheed, and McDonnell-Douglas) and 57 air carriers/organizations were analyzed by NASA. These 
data provide a “lessons learned” knowledge base which may be used for the definition of guidelines for flight deck 
automation and its associated crew interface within the High Speed Research Program. The aircraft chosen for 
analysis represented a progression of levels of automation sophistication and complexity, from "Basic" types (e.g., 
B727, DC9), through "Transition" types (e.g., A300, Concorde), to two levels of glass cockpits (e.g.. Glass 1: e.g., 
A3 10; Glass 2: e.g., B747-400). This paper reports the results of analyses of comments from pilots flying 
commercial transport types having the highest level of automation sophistication (B757/B767, B747-400, and 
A320). Comments were decomposed into five categories relating to: (1) general observations with regard to flight 
deck automation; comments concerning the (2) design and (3) crew understanding of automation and the crew 
interface; (4) crew operations with automation; and (5) personal factors affecting crew/automation interaction. The 
goal of these analyses is to contribute to the definition of guidelines which may be used during design of future 
aircraft flight decks. 


INTRODUCTION 

There has been a significant impact brought about by the progressive introduction of automation technologies 
onto commercial transport flight decks. The majority of these enhancements have been positive. For example, 
automation has allowed more efficient flight path management, a reduction in crew workload resulting in 
certification of the two-pilot cockpit, new crew interface features, and more efficient use of cockpit “real estate.” 
However, human factors problems with ever-more complex flight deck automation have also been noted. Wiener 
and Curry (1980) foreshadowed these problems, while researchers such as Wiener (1989), Sarter and Woods (1991), 
and Norman and Orlady (1989) added to our knowledge. The pilot/automation interface has been implicated in 
accidents involving “glass cockpit” aircraft (e.g., Sparaco, 1994; Mecham, 1994), prompting guidelines for “human- 
centered aircraft automation” (Billings, 1991) and, more recently, a review of automated cockpits and their crew 
interfaces (Hughes and Dornheim, 1995). A systematic analysis of crew experiences with automation —that is, 
“lessons learned” from line experience— may aid in the definition and enhancement of human factors guidelines for 
the design and use of future automated flight decks. 


METHOD 

An Automation Attitudes Survey was distributed by the RAF IAM and IATA to commercial transport pilots 
within the United Kingdom, North America, Europe, the Middle East, Asia, Australia and New Zealand. All 
surveys were confidential. The survey consisted of four sections: biographical information (e.g., age, crew 
position); ten bipolar statements to determine level of experience with glass cockpits; 78 bipolar statements 
addressing specific issues with regard to automation (e.g., training, display usage); and an open section for freehand 
comments. A total of 1,914 completed surveys was returned for analysis. Principal Components Analyses of the 
bipolar statements were performed by the RAF IAM and have been reported in the human factors literature (James, 
McClumpha, Green, Wilson, & Belyavin, 1991; McClumpha, James, Green, & Belyavin, 1991). The present report 
addresses preliminary results of ongoing analyses being performed by NASA of the survey freehand comments. 

Thirteen commercial transport aircraft types were selected for analysis (inclusion of versions within selected 
aircraft types translated into a total of 15 aircraft categories). Four levels of automation complexity were defined 
and the 15 aircraft were ranked according to these definitions. The first automation complexity level was “Basic” 
(e.g., B727; DC9). A Basic cockpit has only simple flight instruments and rudimentary navigational instruments 
which are not generally integrated in display. There is an autopilot and a flight director. The orientation and 
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monitoring of the flight profile is done solely by the pilot and simple computer assistance is provided in the form of 
steering information for the pilot to perform The second automation complexity level, “Transition” (B737-300; 
A300; DC 10), implies a cockpit with an onboard navigational system integrated into a computer-based flight profile 
monitor. The navigation system may be ground-based, OMEGA, VOR/DME (Very high frequency 
Omnirange/distance measuring equipment), or inertial; however, the autopilot follows the navigational instructions 
and usually is able to autocouple to an ILS (Instrument Landing System) at Category II (ILS approach procedure to 
a height above touchdown of not less than 100 feet, with runway visual range not less than 1,200 feet) as a minimum 
approach aid. Vertical navigation guidance is provided to fly a more efficient flight profile. There were two “glass” 
levels of automation complexity. “Glass 1” cockpits (e.g., A3 10; B737-400) have a fully-automated flight system 
with CRT (cathode ray tube) displays of flight instruments and navigational information. Navigation and flight 
profile are programmable and automatically monitored, with the autopilot flying most of the time. This cockpit has 
a full EFIS (Electronic Flight Instrumentation System) and FMS (Flight Management System). The autoland 
capability is usually Category IIIB (ILS approach procedure without a decision height maximum and with runway 
visual range not less than 150 feet), often with roll-out and low visibility takeoff guidance. The highest level of 
automation complexity, “Glass 2” (e.g., A320, B757) includes all of the Glass 1 features with the addition of 
systems automation (e.g., EICAS [Engine Indication Caution and Alerting System] and ECAM [Electronic 
Centralized Aircraft Monitor]). 

Freehand comments were analyzed within each aircraft category and were collapsed within a single automation 
complexity level. The numbers of respondents and respondents providing comments (and, therefore, serving as the 
basis for these comments analyses) by aircraft type and level of automation complexity are provided in Table 1. For 
the purposes of brevity, this report focuses on the results of analyses of freehand comments provided by pilots flying 
the highest level of automation complexity. Glass 2 (a detailed description of all comments analyses is in 
preparation). The Glass 2 aircraft analyses are based on a corpus of comments from 443 respondents (56 + 107 + 
280) flying three aircraft types (A320, B747-400, B757/767). Respondents included Captains (N=249), First 
Officers (N=175), and Second Officers/Flight Engineers (N=12) working for 20 air carriers (two respondents were 
retired and five gave no crew position). Pilots flying Glass 2 category aircraft had a mean of 1161 hours on their 
respective types with a mean of 9378 total flight hours. Mean age was 42 years with a mean of 20 years of 
professional flying. Therefore, analyses of crew comments within the Glass 2 category were based on all crew 
positions with pilots having broad experience across a number of airlines. 

Table 1 

Numbers of Respondents & Respondents Providing Comments by Aircraft Types & Automation Complexity Level 



Respondents 

Respondents providing comments 


(N) 

(N) 

Basic 

B727 / 727-200 

16 

8 

B737-200 

85 

25 

DC9 

14 

3 

Transition 

A300 

13 

10 

B737-300 

153 

103 

B747-100 / 200 / 300 

148 

72 

DC10 

22 

10 

Lion 

61 

34 

Concorde 

7 

3 

Glass 1 

A310 

102 

34 

B737-400 

120 

80 

MD-80 

6 

3 

Glass 2 

A320 

83 

56 

B747-400 

176 

107 

B757 / 767 

464 

280 
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A method for systematically categorizing crew comments was developed by reviewing a random sample of 
approximately 400 surveys. Five broad categories of comment types were defined. (1) General comments related 
to such factors as safety and general good/poor things about flight deck automation. (2) Design comments fell into 
two subcategories, those relating to the crew interface and those relating to the underlying philosophy of automation 
design and its use. (3) Comments in the Understanding & Automation Use category involved crew understanding 
of automation and related systems (e.g.. Flight Management System). (4) Crew Operations comments described 
the relationship of automation to crew workload, trust, training, and cockpit operations, and the impact of 
automation on airmanship and skills. (5) Crew Personal Factors comments described the relationship between 
automation and specific crew factors, such as age and experience. 

RESULTS 


General issues related to automation 

With regard to flight safety, the general consensus was that safety is increased with automation. Specifically, 
automation allows more "thinking and monitoring" time, is less fatiguing, and produces a less stressful work 
environment. However, there were specific concerns with automation. Automation may lend a false sense of 
security, particularly with inexperienced pilots. It may also produce a higher sense of "insecurity" during an 
automation failure. Pilots believe there is a general temptation to ignore raw information and "follow the 
green/magenta line." In addition, there was general concern that automation may increase boredom, thereby 
indirectly decreasing safety. Automation was seen as a progressive step in the right direction in that it allows more 
accurate and efficient airline operations. However, several areas of possible improvement were noted. 

Design 

Flight deck designs were considered to be generally good, but design problems with specific aircraft were 
reported. Initial problems with software seem to have been reasonably cured. There is a significant positive 
response to the map display. The typical EFIS presentation is very usable, but it "takes getting used to." Several 
specific problems were noted with regard to displays and crew interaction with automated systems; for example: 
map shift; difficulty with airspeed and altitude tapes; information clutter; presentation of engine secondary data; 
indecipherability of messages (e.g., due to poor wording and poor use of abbreviations); incorrect feedback (e.g., 
fuel predictions); improper signaling (e.g., of V4 climb-out speed); overall lack of feedback from systems; 
irrelevance of some displayed data; slow system activation and response time; extreme complexity with failures; 
and variable reliability and false/spurious warnings of automated systems. With regard to controls and the 
underlying flight control philosophy, the response to fly-by-wire is very positive; however, there is an overall 
negative response to the perceived lack of feedback from the fixed autothrust system and auto-trim (e.g., "the seat of 
the pants is gone...") and lack of cross-cockpit and autopilot control coupling. 

Understanding and Automation Use 

Generally, pilots report an overall positive response to automation elements and their integration. They believe 
automation allows them to concentrate on the "real world outside" and makes aircraft handling easier. In particular, 
the navigation information is considered very good and pilots appreciate the option to "turn off" the automation and 
revert to basic flying skills. However, there were several concerns with regard to the use of the automation. Pilots 
noted that automation requires more self-discipline— it "makes things too easy," and they may find themselves in 
"traps" that lead to accidents and incidents. In glass cockpits, it is easier to be "drawn in" and lose sight of the 
aircraft; that is, it is easier to become a "spectator" and lose awareness of ongoing operations. In particular, careful 
monitoring of mode annunciations is extremely important. Pilots also report feeling somewhat isolated or distanced 
from the physical aircraft. For example, pilots noted great difficulty in anticipating aircraft behavior. In addition, 
the "thought process" of flying is different (i.e., "flying through the computer" rather than directly controlling the 
aircraft) and time is required to adapt to this method. 

Pilots also reported a strong sense of "over-automation," i.e., technology for the sake of technology rather than a 
clear requirement for an automated feature and sufficient consideration of operational and practical experience in its 
design. They report that interactions with the FMS, in particular, are very complex. Pilots find FMS programming 
to be time-consuming and that the automation cannot deal adequately with ATC changes. Some reported lack of 
confidence in their ability to program the FMS; "what's it doing now" is still common, although it tends to happen 
primarily during the first six months of line flying. Pilots also believe the crew role is being redefined by 
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automation; they are now "monitors" — a role with which they are uncomfortable and for which they were not 
trained. They believe that automation is a tool that should complement the crew and must be under crew control; 
they object strongly to company-mandated use of automation. There is also a concern with preoccupation with the 
FMS during flight — the "both heads in the cockpit" syndrome. This appears to be more pronounced with less 
experienced pilots. Some pilots proposed that "FMS fiddling" be restricted below 10,000 feet. Also, pilots believe 
that the notion "one pilot handles the FMS, the other handles the aircraft," should be emphasized in crew procedures 
and during training, because of the prevalence of this problem. 

Impact of automation on crew operations 

As has been reported in earlier studies (e.g., Weiner, 1989), automation may reduce workload in low workload 
flight phases and may increase workload in high workload flight phases; also, workload may be increased 
dramatically during abnormal situations and failures. Navigation capability and associated displays were noted as 
major workload reducers. Because of the high workload associated with interacting with the FMS, in particular, 
some crews have adopted the practice of predicting ATC (Air Traffic Control) instructions and "preprogramming" 
them into the FMS to reduce workload during final approach, even though this is not dictated by company SOPs 
(standard operating procedures). A number of pilots noted that it is often easier to just turn off the automation and 
"go manual" because of the workload penalty paid by the complexity of interactions with the automated systems. 

With regard to trust in automation, pilots have some reservations. They report good, but variable, reliability; i.e., 
some automated systems are reliable; others are not. However, their primary concern is that many pilots report 
observing colleagues who become complacent and rely too much on the automation. A number of reasons were 
given for this "complacency": some pilots are "lazy" or over-confident in their ability to operate the equipment; 
some company policies and SOPs mandate reliance on automation; some training programs encourage pilots to rely 
on automation and may reduce individual initiative; and some training programs do not emphasize enough the need 
to cross-check raw data. 

Several comments were made relating to crew training and automation. Many pilots noted that the initial training 
program was very difficult and they felt unprepared for their first unsupervised flight. Insufficient information was 
provided concerning the actual aircraft and its systems, making it difficult to assess potential problems. They 
wished for more hands-on practice and more simulator time, to consolidate their knowledge. Some believe the 
training philosophy needs to be refocused and that attention should be given to the basics of flying and automation 
should be placed in this context, instead of concentrating primarily on the automated systems. Conversions "up" to 
automated aircraft were not a problem (other than those associated with the amount of information and the new 
approach to flying); conversions "down" were disconcerting— scan and basic manual skills deteriorated or changed 
with automation use. Pilots reported a long learning curve: six months to one year of line operations is required to 
fully understand the FMS. Also, the regulatory authority requirements during periodic checks do not reflect the 
reality of operating automated cockpits. A primary concern expressed is that present training methods may produce 
an "era of button-pushers," not pilots. 

With regard to cockpit operations, there is a belief that major incidents were primarily due to poor flight deck 
communication, a condition certainly not unique to automated flight decks. It is reported, however, that there is an 
enhanced need with automation for crew discipline, a clear division of duties, and maintenance of both pilots in the 
loop. With automation, crew communication tends to be more visual, less verbal. That is, communication is often 
dependent on the automated system visually reflecting a crew member's input; therefore, communication can be 
diminished if a crew member's interaction with an automated system is not mirrored. Some pilots felt that there 
was an ongoing redefinition of crew roles and that "authority for manipulating the aircraft can shift subtly" based 
upon which crew member was actually operating the FMS. They believe that new SOPs are required to utilize the 
unique capabilities of the automated flight deck. Pilots report two problems when interacting with ATC: first, ATC 
does not adequately use automated aircraft capabilities and, second, ATC needs to appreciate the difficulty and 
workload involved in requiring the crew to reprogram the FMS. 

Automation may reduce crew airmanship and basic skills, even though, "Automation has not changed the 
fundamentals of airmanship. Keeping air under the wings, knowing your position in space, being in the right 
configuration, are still as important as ever." Pilots reported a decrease in confidence in their handling abilities, a 
loss of (or change in) scan, and a decrease in navigation/position awareness. Many pilots reported increased self- 
discipline in "turning off automation" and regularly practicing manual flying. They believe management should 
encourage this behavior and must not mandate use of automation over hand-flying. 
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Crew Personal Factors 


Pilots believe that automated aircraft are generally less stressful and less fatiguing to fly (although there may be 
increased boredom, which in itself may cause problems). As noted earlier, the need for self-discipline is enhanced 
because the automation may lull you into a false sense of security. The primary concern with regard to crew 
personal factors involved experience level: more experienced pilots are apprehensive with the placement of 
inexperienced pilots on highly automated flight decks. Also, a number of experienced pilots commented that 
younger, inexperienced pilots are "computer keen," that they "fixate" on the automation, and their lack of experience 
prevents them from "knowing when to throw it away." They recognize that younger pilots grasp the fundamentals 
of automation faster than older pilots because of their general computer experience, but their lack of experience with 
conventional aircraft provides no knowledge base to rely upon in the event of an automation failure. Experienced 
pilots maintain that training pressures may not allow the development of basic piloting skills and also note that 
inexperienced pilots appear less aware of such flight fundamentals as airspeed, altitudes, and navigation. 

EXAMPLE PROPOSED GUIDELINES 

The goal of this research is to contribute to the definition of human factors guidelines which may be used in the 
design of future automated flight decks and the context within which they are operated. Example guidelines, 
derived from the results reported above, follow. It should be noted that some of these guidelines may already be 
implemented within present automated flight decks. In addition, these proposed guidelines were derived from pilot 
opinion and need to be validated prior to their application. 

Design 

• Pilots believe that development of “perfect” software with complex software-based systems is not possible. 
Consideration should be given to “not automating” some functions (even if the functions are “automatable”) to 
lower risks associated with these types of systems. 

• Review the design of the map display and enhance it; it is widely used by crews. Ensure the map display 
correctly reflects present position (i.e., solve the map shift problem). 

• Provide a human factors review of the following and enhance the design: airspeed tape, altimeter tape, 

information clutter, engine secondary data presentation, and system messages. 

• Review the overall system feedback scheme; identify and correct incorrect feedback and improper signaling. 

• Failures of automated systems are complex and difficult for crews to manage. Simplify the interface such that it 
aids crews in decision making and in rectifying the problem. 

• Provide physical feedback with, and cross-cockpit coupling of, flight controls. When considering “passenger 
comfort” in the design of automated flight controls, do not remove the “seat of the pants” feeling, but design the 
automated flight controls such that the pilot feels “attached” to the aircraft. 

Understanding and Automation Use 

• Allow pilots to easily “turn off’ the automation and revert to manual flying, and make this choice clearly 
apparent to the crew. 

• Do not automate a function for the sake of technology; reference automation design to crew needs and 
operational and practical airline experience; justify all automation features. Understand that each automated feature 
must be trained and completely understood for its correct use. 

• Automation should be a tool to be used by the crew and it should complement crew skills; automation should 
always be under crew control. 

• Consider restricting crew interaction with the FMS to above 10,000 feet. 

Impact of automation on crew operations 

• Recognizing that automation may increase workload during high workload phases of flight, develop SOPs that 
address this problem (e.g., suggest information to be preprogrammed into the FMS prior to approach in preparation 
for a change in ATC instructions). Allow pilots to turn off the automation during high workload periods. 

• Enhance crew training with regard to automation: describe the self-discipline required; focus on maintaining 
operational awareness; emphasize careful monitoring of modes; emphasize the “thought process” associated with 
flying automated aircraft (e.g., “flying through the computer”). Extend crew transition training to glass cockpits to 
allow more time to absorb such complex information. Provide more information concerning the aircraft and aircraft 
systems. Provide more hands-on training with the automated systems; allow use of the simulator for consolidating 
learning. Develop training procedures and SOPs to correct the problem of complacency. Emphasize during training 
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the need to not “fixate” on the automation, but to actively monitor it and cross-check to raw data. Automation use 
should be taught within a “basics of flying” context, rather than concentrating purely on the automated systems. 
Review the format of periodic checks and update them to the reality of glass cockpit line operations. 

• Enhance crew awareness of the need for crew discipline, a clear division of duties, and maintenance of both 
pilots in the loop. Develop SOPs to support these goals. Design displays to clearly mirror crew inputs, 
understanding that much of crew communication in automated cockpits is visual. Develop new SOPs to fully utilize 
capabilities of the automated flight deck. Enhance ATC awareness of crew operations in glass cockpits. 

• Enhance crew awareness of the potential reduction of basic airmanship and skills. Encourage crews to regularly 
practice manual flying and develop supporting SOPs. 

Crew Personal Factors 

• Provide guidelines to crews to support them in dealing with boredom in automated cockpits. 

• Review the rules governing assignment of ab initio or inexperienced pilots to glass cockpits; consider providing 
a mechanism to allow inexperienced pilots to develop a firm base of piloting skills while operating highly 
automated aircraft. 


SUMMARY AND CONCLUSIONS 

In summary, we as a community now have several years of commercial transport experience with highly 
automated flight decks. A systematic review of line pilots’ experiences with, and attitudes about, flight deck 
automation may be used to contribute to the design and development of future human-centered automated flight 
decks and a supportive operations context. 
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